Network-based rendering and steering of visual effects

ABSTRACT

Systems and methods for applying visual effects to video in a client/server or cloud-based system are described. Using a web browser, for example, users can apply and control both simple and sophisticated effects with dynamic real-time previews without downloading client-side software. Rendering and previewing of the effects is done on the server and rendered images can be sent in rapid succession to the client, allowing simple, real-time feedback and control. Other aspects of the technology are also described.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This is a continuation of U.S. patent application Ser. No. 14/349,178,entitled “Network-Based Rendering And Steering Of Visual Effects,” filedon Apr. 2, 2014 (Attorney Docket No. 114641.8002.US01) which claimspriority to PCT Patent Application No. PCT/US12/059572, entitled“Network-Based Rendering And Steering Of Visual Effects,” filed on Oct.10, 2012 (Attorney Docket No. 114641.8002.WO00) which claims priority toU.S. Provisional Patent Application No. 61/545,330, entitled “Method ForNetwork-Based Rendering And Steering Of Visual Effects,” filed on Oct.10, 2011 (Attorney Docket No. 114641.8002.US00), each of which is hereinincorporated by reference.

BACKGROUND

Visual effects include the various processes by which imagery is createdand/or manipulated on film and/or video data. Visual effects are oftenimplemented digitally by processing successive frames of video orscanned film footage. Each frame or set of frames of digitized video isprocessed by computer algorithms which produce output framescorresponding to the input frames, processed by the effect(s) inquestion. A collection of one or more effects is defined herein as alook.

Visual effects are usually applied in a computer system, where framesare read from a disk or storage device into the computer, the frames areprocessed, and written to the same or a different storage device. Theeffects are controlled either interactively by a user (the videocreator), or in batch using a predefined script.

In the interactive case, the video creator uses a “host” applicationsuch as Adobe After Effects, Autodesk Flame, Avid Media Composer orsimilar video editing or compositing applications to (a) identify theframe sequence to apply the effects to, (b) select the effects, (c) setthe various parameters which control the final look of the effect, and(d) render the effects by performing the image-processing algorithms onall frames in the sequence. Controls might include blur amount, imageshift, strength of color correction, position to apply an overlay image,and many others, as required by the look.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating an environment in which thedisclosed technology may operate.

FIG. 2 is a display page illustrating a user interface through which auser can apply effects to a video clip and make adjustments to the videoclip and/or the effects.

FIG. 3 is a block diagram illustrating a software architecture that maybe employed by the disclosed technology.

FIG. 4 is a timing diagram illustrating a transaction between a clientbrowser and various components of a server.

FIG. 5 is a flow diagram illustrating the processing of an interactivesession.

FIG. 6A is a block diagram illustrating an environment in which thedisclosed technology may operate and that includes a video editor andeffects server that are co-located.

FIG. 6B is a block diagram illustrating an environment in which thedisclosed technology may operate and that includes a video editor and aneffects server located in different networks.

DETAILED DESCRIPTION Visual Effects

Various examples of the applicant's invention will now be described. Thefollowing description provides certain specific details for a thoroughunderstanding and enabling description of these examples. One skilled inthe relevant technology will understand, however, that the invention maybe practiced without many of these details. Likewise, one skilled in therelevant technology will also understand that the invention may includemany other obvious features not described in detail herein.Additionally, some well-known structures or functions may not be shownor described in detail below, to avoid unnecessarily obscuring therelevant descriptions of the various examples.

The terminology used below is to be interpreted in its broadestreasonable manner, even though it is being used in conjunction with adetailed description of certain specific examples of the invention.Indeed, certain terms may even be emphasized below; however, anyterminology intended to be interpreted in any restricted manner will beovertly and specifically defined as such in below.

Client/Server Video

Increasingly video creators are uploading videos to cloud-based webservers where the videos are stored and streamed to users worldwide;examples include Vimeo, YouTube, and many others. These cloud-basedsystems allow creators and developers to take advantage of distributedhardware and/or software computing resources provided as a service overa network (e.g., the Internet). Editing and compositing of video usingonline applications is only beginning to be done with systems such asClesh, YouTube Video Editor, and Kaltura. The present system, describedin detail herein, provides a mechanism to add sophisticated visualeffects capabilities to those systems as a natural part of an onlinecreation and editing process. Current online and cloud-based videoediting systems either do not include significant effects, or includeonly very simplistic or low-quality effects which do not require evenseconds of rendering time nor provide the ability for users to controlor steer them.

The disclosed techniques seek to improve the situation described above.Effects processed on a client computer require a video to be downloadedto the client and re-uploaded when complete, which is inefficientcompared to the methods in the present system. Techniques for processingeffectively and efficiently on a cloud-based server while using standardsoftware and hardware components in client devices rely on asophisticated set of interactions and data exchange between client andserver, such as the systems and methods described herein.

Systems and methods for applying and controlling or steering theapplication of high-quality effects to video in a client/server orcloud-based system are disclosed. High-quality effects (e.g., blurs,warping, rays, and shake reduction) are commonly more sophisticated andtake more computation time per frame than lower quality effects (e.g.,simple color transformations), and often have more controllability. Somedistinguishing features of high quality effects are that the underlyingcomputer algorithms are antialiased (no “stair stepping” or“pixelation”), use a nearly unlimited color space such as floating pointarithmetic, provide for accurate color rendition, and because of theadvanced computational algorithms, produce a result which is morerealistic looking than lower quality algorithms. Advanced computer codesuch as that used in high quality effects often enables more flexiblecontrol of the looks of the effects, adding to the realism and thus theperceived visual quality.

In one embodiment, seen in FIG. 1, the video creator Desktop User A(100) uses a desktop computer (101) with a standard web browser such asFirefox, Internet Explorer, or Google Chrome (102). The browser connectsto a cloud server farm (105) via the Internet or any similar networksuch as a mobile data network or private network (103). The user isable, by means of a web page running in the browser, to apply effects tosets of video frames, and adjust the controls of those effectsdynamically, in real time. For example, the effects may include lensflares, shake addition, old-film styles, old-TV styles, transitions,stylistic renderings (halftone, sketch, oil paint, etc.), cartoon looks,glows, video-feedback, motion blur, frame rate conversion, sharpening,vignettes, kaleidoscopes, embossing, adding textures, simulated naturalphenomena (lightning, fire, smoke, rain, snow, northern lights,rainbows, etc.), and so on. The present system allows for the user tosee the effects of their adjustments immediately (or near real-time), aswell as initiating and controlling batch renders of large numbers ofvideo frames. The video creator user (100) may also be using a mobiledevice (104) with a browser in a similar fashion.

Referring to FIG. 2, in one embodiment, the user uses a standard webbrowser (201) to apply effects to a video clip, and makes adjustmentsdynamically (at will). The video appears in the Video Preview Window(202), and can be played using the Play Button (203 a, 203 b). Effectsappear in the Thumbnail Panel (204); the user may choose one or moreeffects from that panel to be applied to the video clip (205). To adjustor steer the effect, one or more Effect Control Sliders (206) areprovided; the user can drag them and see the results in real time in theVideo Preview Window. When the user is satisfied with the effect, he mayrender the completed video using the Render Video button (207).Alternatively, the system may speculatively render all or part of thevideo without user intervention. In some cases, the user can drag theEffect Control Sliders as the video is being played, and the alteredeffect(s) will be visible immediately, or within a short delay, as thevideo plays, giving the user real-time feedback on the changes effectedby the control.

The Software Architecture Diagram of FIG. 3 describes one possibleoverall structure of the application (though not the only one possible;others are clearly evident to persons skilled in the art). The client(301) (which may be running on a desktop and/or mobile device) connectsvia a set of standard Internet protocols such as TCP/IP and HTTP throughits web browser or similar application to a remote server program (302)running on one or more server computers. The server process or processescontain functions and/or modules to generate web page content (303),create and manage the user interface for rendering and steering theeffects (304), a standard web server (307) such as Apache or MicrosoftInternet Information Server or other, and the Render Library (305) whichcontains the effect processing and rendering code. It may also containvarious other modules as part of normal operation, such as a Webframework (306). It is a feature of this system that the client userdoes not have to install any special software to perform theapplication, beyond the usual operating system and web browser alreadyfound on most modern computers.

The HTML/CSS page generation code (303) is used to deliver the web page(HTML and CSS) to the browser client, as well as JavaScript (JS) code tobe executed in the browser. The HTML/CSS page generation code may beimplemented using static HTML or any web server-side language such asPHP, JavaScript, Java, Python or others. The Render interface code (304)is used as an intermediary between the browser (301) and the RenderLibrary code (305) and responds to asynchronous requests from thebrowser, manages the Render Library code, and formats messages fordelivery to the browser. The Render Library code (305) implements theprocessing of the effects and processes the video pixel data for eachframe, using the selected effect and parameters provided by the user.

Referring to the Interaction diagram of FIG. 4, which describes apossible timeline of events, the first column shows events happening inthe browser. The three columns to the right show three distinct parts ofthe server software. This diagram is only for illustration; many othersimilar embodiments are possible, for instance using a different numberof server processes, omitting the render interface code and folding itinto the render library code, or different styles and numbers ofmessages exchanged.

The browser (301) initiates the conversation with the server code (307),such as via an HTTP GET command (401). The server returns (402) code(e.g., HTML+CSS+JavaScript) which is interpreted by the browser to showthe user's videos and a selection of possible looks to apply. The userindicates, such as by pointing, drag-and-drop or similar means, theintent to apply a particular look to a particular set of video frames.Messages (403) are sent to the server, which collects the video frames,invokes the Render Library software to apply the looks, and optionallyone more text or binary preset files describing the look to be appliedand various metadata about that look (404, 405). It then renders one ormore test frames which are sent to the user (406), which can occurnearly immediately. The server also sends to the browser in (406) adescription of the controls available to steer or adjust the look. Thebrowser interprets that description and displays the controls, which maytake the form of knobs, buttons, sliders, menus or other commoncontrols, as well as on-screen direct manipulation using mouse or touchscreen input, for instance to indicate an X,Y spatial position relativeto the video frame, or to adjust or control a parameter value. Forexample, the parameter values may include brightness, saturation,strength, position (e.g., for an X/Y control), width, length, quality,zoom, color (e.g., with a color chooser or color picker), detail, speed,randomness, tint, blur amount, mix, enable (e.g., yes/no checkbox),angle, combine, threshold, invert, opacity, reverse, variation, style,sharpen, and so on.

The user may then interact (407) in real time with those sliders orcontrols; each update of a control is sent to the server (408), whichrenders one or more frames (409) and sends them immediately to thebrowser (410, 411, 412), providing nearly instantaneous feedback to theuser about the effect of that control. This interaction occurs rapidlyso the video frame appears to reflect the result of the change in theslider or control as the user moves it. The entire loop (412) consistingof steps 407-412 is repeated rapidly.

A sample interaction between a client and a server is provided below:

-   -   User opens browser and selects URL of video server (e.g.:        http://video.genarts.com) (401)    -   Server responds with initial web page contents (402), including        HTML text, images, cascading style sheets (CSS) styling, and        JavaScript (JS) or other scripting language code to run in the        browser.    -   The JavaScript (JS) code asynchronously sends information to the        server to set up the current effect and any preset parameters.        (403)    -   The render code on the server initializes the effect rendering        library, sets the library up to render that effect with given        parameters, and creates the user-visible controls from the        control definitions (404, 405, 406 a, 406 b)    -   The remainder of the interaction happens, such as in a very        rapid loop (413):        -   The user drags a scrollbar in the browser web page (407)        -   The JS code is notified of the user's action and initiates            an asynchronous server request, telling the server to render            the current frame of video with the newly modified parameter            represented by the scrollbar value (408)        -   The server receives the new parameter data and renders the            frame in memory (409, 410)        -   The server builds an HTTP reply packet using the in-memory            image data and any other metadata; this packet can consist            of multiple elements including the image data (PNG format in            this example) (411)        -   The reply packet is delivered to the browser's JS code,            which causes the browser to render the new image (412). The            result, to the user, is dynamic near-real-time control and            updating of the image or video.

The system described herein typically works within a larger videostorage, transcoding and playback system. To explain this context, anexample of an interaction session is provided (for illustration only),referencing the flowchart in FIG. 5:

-   -   User takes video footage with camera or mobile device (500) with        a video capture source    -   User uses web browser to upload raw video to server (501)    -   Web browser presents user's videos and offers editing and        effects processing modes (502)    -   User selects a number of clips to combine into a sequence, trims        to suit, and then begins to add video effects and transitions        (503).    -   At this point (504) the system described herein, utilizing the        software architecture in FIG. 3 and the messages and data        exchanges as illustrated in FIG. 4, allows the user to add        effects in the manner described herein.    -   If the user is satisfied (505) with the result, either: final        rendering happens automatically in the background (without user        interaction) or the user explicitly signals server to begin        final render (506).    -   After rendering is complete, the processed video is ready for        viewing and optional download or distribution (507).

Additional Features/Embodiments

At any time the user can play back the video, or search within it usingvarious standard timeline techniques such as a time bar indicating theposition of the virtual play head. At all times the user can see thecurrent effect(s) applied to the video.

In the case where the desired effect involves a transition between videoclips, the user may select a transition point (a cut between two videosegments) and then the server (again through the browser) reveals aselection of possible transition effects. The user chooses one and canimmediately see the results, using the method described herein.

Note that optional “advanced” modes may be available with slider and/ornumeric parameters to steer or control the effect, in the ways describedabove.

-   -   Whenever the user selects a new look or transition or adjusts        any parameter, the server can be immediately notified using the        methods of this system and the server renders the selected        effects onto the video, possibly using caching and lookahead        methods to avoid excess computation and improve interactive        response to provide a near immediate response. Caching is        achieved by storing previously rendered frames in memory or on        disk, and associating those frames with appropriate parameter        values in such a way as to be able to quickly recall those        frames if the user requests the same frame with the same        parameter values in the future. Lookahead is used to        speculatively render ahead of the current play head position,        anticipating that the user will likely want to see those frames.        Various statistical or heuristic methods for this can be used,        such as detecting idle server capacity and rendering one or two        seconds ahead using the idle CPUs.    -   Note that the server may continuously be streaming processed        video to the client during this process, so the user can also        use this system in a mode whereby s/he is effectively altering        the processing of a running video stream as it is streamed. In        this mode, there is a continuous feedback loop, where the client        provides feedback to the cloud and receives updated video in        (near) real time, which allows the user to make finely        controlled adjustments and see the results of those adjustments        quickly. Nothing in the present system requires processing the        same frame repeatedly, so this is a natural enhancement. The        messaging and data flow in FIG. 4 does not materially change,        although the effect messages become interleaved with video        frames being played out.    -   Note that, especially in the case where the server is        continuously streaming the processed video, the server may        record the slider control values at the time when each video        frame time is being displayed, in such a way as to be able to        reproduce the changes in the slider position and thus the        related parameters at a later time, or during rendering. This        allows the user to animate the slider values as the video is        playing, so as to (for example) fade the effect in or out.    -   Note that the user may be using a browser on a mobile device,        such as a phone or tablet. This system permits use on a less        capable or smaller client device, compared to rendering effects        directly on the device, since most of the detailed image        processing is in the cloud (e.g., one or more devices remote        from the mobile device). It also permits much more        computationally complex effects and effects with higher visual        fidelity, which are more computationally expensive, while        maintaining a fixed frame rate to the client. More server        resources can be dynamically allocated to compute effects of        arbitrary complexity while the client remains very simple, with        low resource requirements.    -   Note that the user may at any time request the server to render        the entire sequence of frames in a batch mode, with later        notification. For example, the user may elect to be notified        upon render completion via text message, popup window, or other        means. The notification may include the URL to access the        modified video. This may be especially useful on a mobile        device, where the user uploads a video taken on the phone, and        does not have to use the device's CPU (and thus its battery        power) to process the video. They can turn off the device        entirely and wait for the completion notification.    -   Users may also process the same video with different effects and        compare the results either side by side or by using analytics to        measure audience engagement. This can be done in the browser by,        for example, selecting an applied look and replacing it with an        alternate look, selected either by the user or recommended by an        algorithm on the server. Two or more versions of the video could        be shown to viewers either side-by-side or sequentially, with a        method (such as a button in the browser) to select which version        the viewer prefers. Alternatively, different versions of the        video with different effects applied could be shown to different        users to measure the users' preferences for certain effects.    -   Users may also process several videos using the same set of        effects; a feature of this system would enable users to select        one or more looks and transitions applied to a set of video        clips, store them for later use, and apply the same set of looks        and transitions to a different set of video clips. For example,        in an advertising scenario, many advertisements may share the        same basic template. This feature would allow users to quickly        and easily apply the same set of looks to all of the        advertisements, to give them a consistent look, thus        communicating the desired branding.    -   Note that the fact that the video frames are rendered on the        server rather than in the user's browser generally means much        more computational power is available for such rendering; this        allows effects of much greater complexity and beauty to be used,        compared with systems which render frames directly in the        browser or on a mobile device. In one embodiment, the video        frames are distributed to multiple physical servers for        rendering, decreasing the total time to render the entire        sequence. This also makes the processing/rendering time        independent of the processing power of the user's device, so a        simple mobile phone can create very complex effects. Different        servers might be made up of hardware with different        capabilities, such as graphics processing units, faster floating        point hardware, or increased memory, which would enable those        servers to be used to process particular effects which took        advantage of the capabilities of those servers.    -   In another embodiment of the system, rendering and playback        tasks can be split between client and server. The client may        render less-complex effects, or low resolution proxy images,        while sending instructions to the server to render higher        quality images asynchronously. These higher quality images, when        ready, can replace the lower quality ones rendered by the        device. Alternatively, in a restricted bandwidth situation, the        device may do local playback of cached rendered frames rather        than constantly streaming from the server.    -   In another embodiment, referencing FIG. 6, an online        (cloud-based) video editing or compositing application 601 may        exist between the user (100) and the effects server (104)        described herein, managing the user's interaction and passing        instructions to and from the server through an API. In this        embodiment, the online application may send instructions and        data and receive instructions and data from the server in        interactive or batch modes, in concert with or independently        from the user's interaction with the editing online application.        FIG. 6 a shows an embodiment with the video editor and effects        server co-located, and FIG. 6 b shows an embodiment with the        video editor and effects server on different networks.    -   In some examples, fidelity of the browser images may depend on        the bandwidth between the client (browser) and server. Higher        bandwidth connections allow both higher quality (containing        fewer compression artifacts) and higher resolution images, and        faster frame rates (number of frames per second). These        variations in image fidelity could be adjusted dynamically and        automatically by the system measuring the connection bandwidth,        and/or can be specified by the user up to the limits of the        physical devices. Note that the resolution of the local device        does not affect final output resolution or quality. The server        may render low-quality images as previews to send to the user's        device, while simultaneously or sequentially rendering        high-quality images for final output. Such resolution switching        could be used, for example, to send lower resolution images to a        small mobile device and high resolutions to a desktop, or to        send low resolution or low-quality highly compressed previews        while adjusting the effect to achieve lower latency feedback.        The server can perform this switching based on querying the        device and network capabilities, without user intervention.    -   Fidelity of the final rendered videos may depend on server        resources. In some cases, users may select quality levels        appropriate to their usage model, which could include paying        more for more computational resources, either to improve quality        (e.g. resolution) or shorten processing time by using more        server processors.    -   Use of standard technology such as the Apache or IIS web        servers, PHP or Python computer languages, and/or web browsers        such as Internet Explorer, Firefox or Chrome allows an        innovative user experience without users needing to download and        install custom applications, namely the highly interactive        steering of effects running on the server.    -   When the user is not interacting with the browser and the server        is idle, the server may render other frames of the video        speculatively, to improve interactive response in the case that        the user desires to see other nearby (typically subsequent)        frames of the video using the current parameter settings. Those        cached frames may be kept or deleted when the user subsequently        changes other parameter settings. The server may be running        multiple processing threads, so as to maximize utilization. The        speculative rendering threads would have lower priority, so they        run only when nothing with a higher priority needs to be done.    -   The techniques disclosed herein may be applied to audio effects,        wherein the audio is streamed from the server and the user        manipulates various parameters of the audio effects such as        reverberation, filtering, noise reduction and gain while the        audio is playing or while a short sample is looped along with        adjustable or controllable parameters, such as octaves,        amplitude, frequency, and so on. This would involve replacing        the core server rendering library with a similar library for        creating and rendering audio effects and extending the data        transfer protocols, while keeping most of the other parts of the        system unchanged.    -   The techniques disclosed herein may be applied to 3-d rendering        and modeling of realistic and/or nonrealistic scenes for use in        films or videos, games, architectural rendering, or other uses.        The user can control via the browser the creation, modeling,        animation, and rendering of such scenes, while the actual        modeling, animation and rendering occur on a server or servers.        This would involve replacing the core server rendering library        with a similar library for modeling, animating and/or rendering        3-d scenes and extending the data transfer protocols, while        keeping most of the other parts of the system unchanged.

CONCLUSION

Those skilled in the relevant art will appreciate that the invention canbe practiced with various computer system configurations, includingInternet appliances, hand-held devices, wearable computers, cellular ormobile phones, multi-processor systems, microprocessor-based orprogrammable consumer electronics, set-top boxes, network PCs,mini-computers, mainframe computers and the like. The invention can beembodied in a special purpose computer or data processor that isspecifically programmed, configured or constructed to perform one ormore of the computer-executable instructions explained in detail below.Indeed, the term “computer”, as used generally herein, refers to any ofthe above devices, as well as any data processor or any device capableof communicating with a network, including consumer electronic goodssuch as game devices, mobile devices (e.g., mobile phones), cameras,video capture devices or source, or other electronic devices having aprocessor and other components, e.g., network communication circuitry.

Aspects of the invention described below may be stored or distributed oncomputer-readable media, including magnetic and optically readable andremovable computer discs, stored as firmware in chips (e.g., EEPROMchips), as well as distributed electronically over the Internet or overother networks (including wireless networks). Data structures andtransmission of data particular to aspects of the invention are alsoencompassed within the scope of the invention.

In general, the detailed description of embodiments of the invention isnot intended to be exhaustive or to limit the invention to the preciseform disclosed above. While specific embodiments of, and examples for,the invention are described above for illustrative purposes, variousequivalent modifications are possible within the scope of the invention,as those skilled in the relevant art will recognize. For example, whileprocesses or blocks are presented in a given order, alternativeembodiments may perform routines having steps, or employ systems havingblocks, in a different order, and some processes or blocks may bedeleted, moved, added, subdivided, combined, and/or modified. Each ofthese processes or blocks may be implemented in a variety of differentways. Also, while processes or blocks are at times shown as beingperformed in series, these processes or blocks may instead be performedin parallel, or may be performed at different times.

Aspects of the invention may be stored or distributed oncomputer-readable media, including computer-readable storage media, suchas magnetically or optically readable computer discs, hard-wired orpreprogrammed chips (e.g., EEPROM semiconductor chips), nanotechnologymemory, biological memory, or other data storage media. Indeed, computerimplemented instructions, data structures, screen displays, and otherdata under aspects of the invention may be distributed over the Internetor over other networks (including wireless networks), on a propagatedsignal on computer-readable propagation or transmission media (e.g., anelectromagnetic wave(s), a sound wave, etc.) over a period of time, orthey may be provided on any analog or digital network (packet switched,circuit switched, or other scheme). Those skilled in the relevant artwill recognize that portions of the invention reside on a servercomputer, while corresponding portions reside on a client computer suchas a mobile or portable device, and thus, while certain hardwareplatforms are described herein, aspects of the invention are equallyapplicable to nodes on a network.

The above detailed description of examples of the invention is notintended to be exhaustive or to limit the invention to the precise formdisclosed above. While specific examples for the invention are describedabove for illustrative purposes, various equivalent modifications arepossible within the scope of the invention, as those skilled in therelevant art will recognize. For example, while processes or steps arepresented in a given order, alternative implementations may beimplemented in a different order, and some processes or steps may bedeleted, moved, added, subdivided, combined, and/or modified to providealternative or subcombinations. Each of these processes or steps may beimplemented in a variety of different ways. Also, while processes orsteps are at times shown as being performed in series, these processesor blocks may instead be performed or implemented in parallel, or may beperformed at different times.

The teachings of the invention provided herein can be applied to othersystems, not necessarily the system described above. The elements andacts of the various examples described above can be combined to providefurther implementations of the invention. Some alternativeimplementations of the invention may include not only additionalelements to those implementations noted above, but also may includefewer elements.

Any patents and applications and other references noted above, includingany that may be listed in accompanying filing papers, are incorporatedherein by reference. Aspects of the invention can be modified, ifnecessary, to employ the systems, functions, and concepts of the variousreferences described above to provide yet further implementations of theinvention.

These and other changes can be made to the invention in light of theabove detailed description. While the above description describescertain examples of the invention, and describes the best modecontemplated, no matter how detailed the above appears in text, theinvention can be practiced in many ways. For example, one of ordinaryskill in the art will understand that, in addition to the visual andaudio effects described herein, the techniques described herein can beused to apply other visual and audio effects to video and audio data.Details of the system may vary considerably in its specificimplementation, while still being encompassed by the invention disclosedherein. As noted above, particular terminology used when describingcertain features or aspects of the invention should not be taken toimply that the terminology is being redefined herein to be restricted toany specific characteristics, features, or aspects of the invention withwhich that terminology is associated. In general, the terms used in thefollowing claims should not be construed to limit the invention to thespecific examples disclosed in the specification, unless the abovedetailed description explicitly defines such terms.

1. A computer-readable hardware device storing instructions that, ifexecuted by a computing system, cause the computing system to performoperations comprising: receiving, from a remote server, a web page,wherein the web page specifies one or more controls for applying visualeffects to video data; providing for display an indication of one ormore visual effects that can be applied to the video data, sending, tothe remote server, an indication of a first selected at least one visualeffect to apply to the video data, wherein the remote server isconfigured to apply the at least one visual effect to the video data andwherein the at least one visual effect has at least one associatedcontrollable parameter, receiving, from the remote server, firstmodified video data, wherein the first modified video data comprises oneor more video frames modified based on the first selected at least onevisual effect; and while the first modified video data is beingdisplayed, receiving an indication to modify the at least onecontrollable parameter associated with the at least one visual effect.2. The computer-readable hardware device of claim 1, the operationsfurther comprising: storing a modified first frame of video data;associating a first value for the at least one controllable parameterwith the stored modified first frame of video data; and storing amodified second frame of video data; associating a second value for theat least one controllable parameter with the stored modified secondframe of video data.
 3. The computer-readable hardware device of claim2, the operations further comprising: in response to receiving a requestto apply the at least one visual effect to the modified first frame withthe at least one controllable parameter set to the first value,retrieving the stored modified first frame of video data; and inresponse to receiving a request to apply the at least one visual effectto the modified second frame with the at least one controllableparameter set to the second value, retrieving the stored modified secondframe of video data.
 4. The computer-readable hardware device of claim1, wherein the video data has associated audio data, the operationsfurther comprising: providing for display an indication of one or moreaudio effects that can be applied to the audio data associated with thevideo data; receiving a selection of at least one audio effect to applyto the audio data associated with the video data; sending, to the remoteserver, an indication of the at least one audio effect to apply to theaudio data associated with the video data; and receiving, from theremote server, modified audio data based on the at least one audioeffect.
 5. The computer-readable hardware device of claim 1, theoperations further comprising: while the first modified video data isbeing displayed, manipulating the display of the one or more controlsfor applying visual effects to the video data based on parameter valuesset for each control and each frame of the modified video data.
 6. Thecomputer-readable hardware device of claim 1, the operations furthercomprising: receiving a second selection of at least one visual effectto apply to the video data, sending, to the remote server, an indicationof the second selected at least one visual effect to apply to the videodata, receiving, from the remote server, second modified video data,providing the first modified video data and the second modified videodata, and receiving a selection of preferred modified video data.
 7. Thecomputer-readable hardware device of claim 1, the operations furthercomprising: while the first modified video data is being displayed,sending, to the remote server, the indication to modify the at least onecontrollable parameter associated with the at least one visual effect.8. The computer-readable hardware device of claim 1, the operationsfurther comprising: while the first modified video data is beingdisplayed, receiving, from the remote server, second modified video databased on the modified at least one controllable parameter associatedwith the at least one visual effect.
 9. A system comprising: a componentconfigured to receive, from a remote client computer, one or more videoframes of video data; a component configured to send, to the remoteclient computer, a web page specifying one or more controls for applyingvisual effects to the video data; a component configured to receive,from a browser executing on the remote client computer, a selection ofat least one visual effect to apply to the video frames, wherein the atleast one visual effect has at least one associated controllableparameter; a component configured to apply the selected at least onevisual effect to the video frames to produce first modified video data;and a component configured to send, to the remote client computer, theproduced first modified video data.
 10. The system of claim 9, furthercomprising: a component configured to identify additional frames of thevideo data; and a component configured to speculatively apply the atleast one visual effect to the identified additional video frames. 11.The system of claim 10, further comprising: a component configured todelete video data in response to determining that the at least onecontrollable parameter associated with the at least one visual effecthas been modified.
 12. The system of claim 9, further comprising: acomponent configured to receive, from the remote client computer, arequest to render the video data based on one or more visual effects;and a component configured to send, in response to determining that thevideo data has been rendered, a notification comprising at least one ofa text message, an email, or a social media posting to a user.
 13. Thesystem of claim 9, further comprising: a component configured to, whilethe first modified video data is being displayed, receive secondmodified video data based on a modified at least one controllableparameter associated with the at least one visual effect.
 14. A method,performed by a computing system having a memory and a processor,comprising: sending, to a remote rendering server, at least a portion ofvideo data comprising one or more video frames; receiving, from theremote rendering server, a web page, wherein the web page specifies oneor more controls for applying visual effects to the video data; and witha browser executing on a processor of the computing system, receiving aselection of at least one visual effect to apply to the video data,wherein the at least one visual effect has at least one associatedcontrollable parameter, and sending, to the remote rendering server, anindication of the selected at least one visual effect to apply to thevideo data, wherein the remote rendering server is configured to applythe at least one visual effect to the video data, and while firstmodified video data is being displayed, sending, to the remote renderingserver, an indication to modify the at least one controllable parameterassociated with the at least one visual effect.
 15. The method of claim14, further comprising: receiving, from the remote rendering server, thefirst modified video data, wherein the first modified video datacomprises one or more video frames modified based on the selected atleast one visual effect.
 16. The method of claim 14 wherein the at leastone visual effect comprises a plurality of visual effects includingblur, warp, rays, shake reduction, color transformation, fade, wipe,biaxial wipe, and still image insertion effect.
 17. The method of claim14, wherein the video data has associated audio data, the method furthercomprising: providing for display an indication of one or more audioeffects that can be applied to the audio data associated with the videodata; and receiving, from the user, a selection of at least one audioeffect to apply to the audio data associated with the video data. 18.The method of claim 17, wherein the video data has associated audiodata, the method further comprising: sending, to the remote renderingserver, an indication of the at least one audio effect to apply to theaudio data associated with the video data; and receiving, from theremote rendering server, modified audio data based on the at least oneaudio effect.
 19. The method of claim 17 wherein the at least one audioeffect is at least one of reverberation, filtering, noise reduction, orgain.
 20. The method of claim 14 wherein the at least one controllableparameter is brightness, saturation, strength, or position.
 21. Themethod of claim 14 wherein the at least one controllable parametercomprises at least one of detail, speed, randomness, tint, blur amount,width, length, quality, zoom, color, mix, enable, angle, combine,invert, opacity, reverse, variation, style, or sharpen.